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Tubule Grading 

This invention relates to a method, an apparatus and a computer program for grading 
tubules In histological slides to provide clinioal information on potentially cancerous tissue 
such as breast cancer tissue, 

5 Breast cancer te a common form of female cancer: Once a lesion indicative of breast 
oancer has been detected, tissue samples are taken and examined by a histopathologic 
to establish a diagnosis, prognosis and treatment plan. However, pathological analysis of 
tissue samples is a time consuming and inaccurate process. It entails Interpretation of 
images by human eye, which is highly subjective: it is characterised by considerable 
0 inaccuracies in observations of the same samples by different observers and even by the 
same observer at different times. For example, two different observers assessing the 
same ten tissue samples may easily give different opinions for three of the slides - 30% 
error. The problem is exacerbated by heterogeneity, i.e. complexity of some tissue sample 
features. 

There is a need tp provide an objective measurement of tubules grading to support a 
pathologist 8 s diagnosis and patients' treatment. 

The presertt invention provides a method of grading tubules in a first image of a 
histological slide characterised in that it has the steps of: 

a) providing a second image of first objects in the first Image which are sufficiently 
farge and of appropriate pixel value characteristics at boundaries to correspond to 
tubules, 

b) providing a third image of second objects in the first image having pbcei value 
characteristics of fat and holes within tubules, 

c) combining data torn the second and third images to identify selected seoond 
objects which are within first objects, 

d) performing one or more of the following: 

i) counting first objects in the first image which correspond to tubules to 
provide a parameter NOB, 

ii) counting the first objects having selected second objects within them to 
provide a parameter N, 

iir) determining the retative area of selected second objects as proportions of 
respective first objects within which they are located to provide parameters 
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RATIO, 

iv) determining Uib total area of selected seoond objects as a proportion of 
total area of first objects within which they are located to provide a 
parameter SURF, 
5 v) determining a parameter PERCENT = N/NOB, and 

vi) counting the number of first objects containing at least medium sized holes 
to provide a parameter T, and 
e) grading the first image's tubules on the basis of the one or more parameters as 
aforesaid with reference to parameter threshold values. 

10 The Invention provides the advantage that it is an objective method of grading tubules in a 
histological slide which does not rely on subjective assessment by clinical personnel. 

The step of providing a second image may incorporate: 

a) thesholding the first image to provide a fourth Image retaining relatively darker 
Image pixels and rejecting relatively lighter Image pixels, 
1 5 b) processing the fourth image by applying the following steps in succession, the first 
being applied to the fourth Image and each after thB first being applied fo the 
outcome of the respective immediately preceding step: inversion, morphological 
dilation, median filtering, hote filling and morphological opening. 

The step of providing a third image may comprise thresholding the first image to provide a 
binary fourth image In which relatively lighter image pixels have a different binary value to 
that of relatively darker image pixels. The step of combining data from the second and 
third images comprises multiplying pixels in the second image by or ANDing them with 
respective corresponding pixels located in like positions the third image. The step of 
grading the first image's tubules employe parameter threshold values set to obtain a 
grading comparable with that obtainable by an appropriate medical expert. 

In a preferred embodiment, the present invention provides a method of grading tubules In 
a first Image of a histological slide characterised in that It has the steps of: 

a) providing a second image of first objects in the first image which are sufficiently 
large and of appropriate pixel value characteristics at boundaries to correspond to 

30 tubules and counting them to provide a parameter NOB, 

b) providing a third image of seoond objects in the first image having pixel value 
characteristics of fat and holes within tubules, 



20 



25 
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c) combining data from the second and third Images to identify selected second 
objects which are within first objects, 

d) counting the first objects having selected second objects within them to provide a 
parameter N, 

5 e) determining the relative area of selected second objects as proportions of 
respective first objects within whfoh they are located to provide parameters RATIO, 
1) determining the total area of selected second objects as a proportion of total area 

of first objects within which they are located to provide a parameter SURF, 
g) determining a parameter PERCENT = N/NQB, 
10 h) counting the number of first objects containing at least medium sized holes to 
provide a parameter T, ancj 
i) grading the first image's tubules on the' basis of the first image's parameters as 
aforesaid with reference to parameter threshold vafues. 

fn this embodiment, the invention provides the advantage that it is an objective method 
1 5 which grades tubules from a variety of parameters yielding multiple grading from which a 
median can be derived. 

in another aspect, the present invention provides computer apparatus for grading tubules 
in a first image of a histological slide characterised in that it is programmed to: 

a) compute a second image of first objects fn the first image which are sufficiently 
20 large and of appropriate pixel value characteristics at boundaries to correspond to 

tubules, 

b) compute a third image of second objects in the first image having pixel value 
characteristics of fat and holes within tubules, 

c) combine data from the second and third Images to identify selected second objects 
25 which are within first objects, 

d) implement one or more of the following: 

i) counting first objects in the first image which correspond to tubules to 

provide a parameter NOB, 
if) counting the first objects having selected second objects within them to 
30 provide a parameter W, 

iii) determining the relative area of selected second objects as proportions of 
respective first objects within which they are located to provide parameters 
RATIO, 

iv) determining the total area of selected second objects as a proportion of 
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V) determining a parameter PERCENT =m/OB, and 

to provide a parameter 7, and 
•) grade the first Image's tubules on th* ho c i c „* *u 

10 drammed * '^ h,te ^ - ' M *M*--*««---telh«. 

and of appropHara pw ^ oharaolBlWtos „, oomZtfM 
tuWee and eoummg mera to provlde „ paramf>ter ^ to """"P 0 """ » 

« ra^w^r^^ inm9flM ^^^^ 

wiaraciBnsucs of fat and holes within tubules 

gj determine a parameter PERCENT = N/NOB 

aforesaid with reference to parameter threshold values. 
»w coniroumg computer apparatus to: 

31 ZZlV™* ,ma ° 8 °' *** «» «*•* ara aufflola* 



•23-pcT-2002 145 31 FROM IP MRLUERN TO UK PATENT 




JH riHLvtRN TO UK EftTE 

• # 



P. 09 



b) compute a third image of second objects in the first image having pixel value 
characteristics of fat and holes within tubules, 

c) combine data from the second and third images to identify selected second objects 
which are within first objects, 

r. 

5 d) implement one or more of the following: 

i) counting first objects in the first image which correspond to tubules to 
provide a parameter NOB, 

10 oounting the first objects having selected second objects within them to 
provide a parameter N, 

10 in > determining the relative area of selected second objects as proportions of 

respective first objects within which they are located to provide parameters 
RATIO, 

rv) determining the total area of selected second objects as a proportion of 
total area of first objects within which they are located to provide a 
15 parameter SURF, 

v) determining a parameter PERCENT = N/NOB, and 

vi) counting the number of first objects containing at least medium sized holes 
to provide a parameter 7, and 

e) grade the first Image's tubules on the basis of the one or more parameters as 
20 aforesaid with reference to parameter threshold values.. 

In a preferred embodiment of this aspect, the present invention provides a computer 
program for grading tubules in a first image of a histological slide characterised In that it 
contains instructions for controlling computer apparatus to; 

a) provide a second Image of first objects in the first Image which are 
25 sufficiently large and of appropriate pixel value characteristics at 

boundaries to correspond to tubules and counting them to provide a 
parameter NOB, 

b) provide a third Image of second objects In the first image having pixel value 
characteristics of fat and holes within tubules, 

30 c ) combine data from the second and third Images to Identify selected second 

objects which are within first objects, 

d) oount the first objects having selected second objects within them to 
provide a parameter N, 

e) determine the relative area of selected second objects as proportions of 
35 respective first objects within which they are located to provide parameters 
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of initial image feature, tubules 51 with pronounced dark boundaries containing one or 
more holes such as 52, large dark objects 53, fat cells 55 with only very narrow faint 
boundaries and small dark objects 56 of no interest. 

Step 32 is shown in more detail as a number of constituent steps a) to e) within chain lines 
5 33. At a), the objective is to select from the grayscale image 50 only relatively darker 
. pixels and omit relatively lighter pixels. An image is obtained from a) which contains 
relatively darker objects of varying size: the larger of these objects are likely to contain 
tubules, but others smaller in size are unlikely to do so. To Implement step a), initially all 
pixels in the greyscale image 50 are compared with one another to obtain their maximum 
10 (Maxg) and minimum (Ming) values: these values are used to compute a parameter P 
given by: 

P m 1 2(Maxg - Ming)/1 00 ^ 

Each pixel in the greyscale image 50 is then divided by 255 so that it lies in the range 0 to 
1 . The value P is then used to transform the grayscale image 50 into an output image as 
15 shown in Table 1 below: 



Table 1 



Input image pixel value 


Output image pixel value 


< 1/255 


0 


> 1/255 AND £ (Maxg - P)/255 


Mapped to the range [0, 1 ] 


> (Maxg - PV255 


1 



This means 1/255 becomes 0, (Maxg - P)/2S5 becomes 1 and x & 1/255 AND s (Maxg - 
PV255 becomes (x-1/Z55y({Maxg-P}/255 - 1/255). The resulting output image values are 

now thresholded to produce a binary image: all output Image pixel values less than a 
threshold value of 0.85 are sBt to zero, and all other pixel values are set to 1. The 
threshold of 0.85 was arrived at experimentally using trial images. 

At b) the image output from step a) Is inverted so that epithelial layers now appear white 
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and background and holes appear black. A morphological dilation Is appOed to the 
inverted Image which thickens and closes smafl gaps in the white epithelial layers but 
leaving background and tubule holes unaffected. The objective of b) is to close 
boundaries likely to contain tubules that might have been left open by step a). 
5 Morphological operations are often used to fuse narrow gaps and eliminate small holes in 
individual groups of contiguous pixels appearing as blobs in an image. They can be 
thought of as removal of irregularities or spatial "noise", and they are standard image 
processing procedures published in Umbaugh S.C., 'Colour vision and image processing', 
Prentice Hall, 1998. tn this Instance toe morphological dilation is represented by a disc- 
1 0 shaped structuring element with 5 rows and 5 columns and corresponding (by convention) 
to a radius R of two pixels: 



0 1 o o 4 

1 1 1 0 T 



0 
0 

Structuring element= 11111 ( 2 ) 
15 0 1110 

00100 

here the expression "disc-shaped" arises from the distribution of 1e in the element which 
appear as a rudimentary disc. 



Morphological dilation Is an expansion operation: for an original binary image (i.e. having 
20 pixel values 1 and 0 only), the expansion operation comprises locating eaoh pixel with 
value 1 and setting pixels nearby to it also to 1. The structuring element defined by 
Equation (2) above illustrates this: the oentral 1 Indicates a pixel under consideration and 
1s above, below, on either side of it and diagonally near to it Indicate pixels in the binary 
Image set to 1; Os En the structuring element Indicate relative positions of pixels that 
25 remain unchanged. 

At c) a median filter is applied to the output of step b): the median filter operation seleots 
each pixel in the output of step b) in succession and takes a 3 x 3 array of those pixels 
centred on the selected pixel; the 3 x 3 array of pixels is then sorted into ascending order 
of pixel value, Le. 0s followed by 1s. The median pixel value (fifth of nine) is then taken as 
30 the filter output to replace the value of the selected pixel. The filter output will be 0 or 1 
according to whether or not the 3 x 3 array has more 0s than 1s. This is repeated across 
the image. Pixels in or separated by one pixel from edge rows and columns do not have 
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the requisite 3x3 array, and for these the original pixel values are retained in the median 
filtered image. Median filtering Is desirable to remove small Isolated groups of pixels 
generated during thresholding at a). Step o) is desirable (but not essential) - it has the 
effect of smoothing blobs In the median filtered binary image produced at b). 



10 



The output Image from step c) contains image regions or blobs which are white outlines 
around tubule structures,, and the next process at d) is to fill these blobs to make them 
solid for further prooessing - the objective Is to fill in holes which ideally lie within tubules. 
To achieve this, all pixels contained within each boundary of white pixels are set to the 
same value as the boundary. Here a hole is a set of background pixels that cannot be 
reached by filling in the background from the edge of the image: the technique of hole- 
filling is based on morphological reconstruction as published in Piere Soille, 
'Morphological image analysis: Principals and Applications', Springer- Veriag, 1999, pp 
173-174. At e), a binary Image resulting from d) is then subjected to morphological 
opening using the 9x9 disc shaped structuring element, defined below in Equation (3). 



15 0 0 1 

0 1 1 

1 1 1 
1 1 1 

Structuring element = 1 
20 1 

1 
0 

0 0 11 



1 
1 
1 
1 

1 1 
1 1 
1 1 
1 1 



1 

1 

1 

1 

1 

1 

1 

1 

1 



0 

1 
1 
1 
1 
1 
1 
1 

0 



0 
0 

1 

1 
1 
1 
1 

0 
0 



(3) 



25 Step e) removes small groups of dark pixels such as 56 that are not part of larger objects 
or boundaries around tubules. 

Steps a) to s) identify blobs in.the greysoale Image 50 (corresponding either to dark areas 
with holes Inside or to wholly dark objects) that are more likely to be the boundaries of 
tubules. In Figure 4, a binary image 60 is shown which is the output of step e): identified 
30 blobs are now represented as closed blobs 61 (I.e. containing no holes) of varying sizes of 
pixel value 1 in a background 62 of 0s. 



Step 34 is a procedure undertaken to extract white regions of the initial greysoale image 
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II 

50, these region* appearing as relatively hfgh value pixel groups. This step is shown in 
more detail at f), g) and h) within chain lines 35, At fl, eaoh pixel value in the initial image 
50 is divided by 255 so that ft «es in the range 0 to 1, and then the pixel values x In the 
resulting image (input image in the table below) are remapped to an output image for the 
seleotion of light regions in accordance with the following table: 



Input image value 


Output image value 


< (maximum - Q ) /255 


0 


(maximum - Q>/255 AND <s i.o 


Mapped to the range [0,1] 



Where Q « 40(Maxg - MingyiOO 



In the above table, the expression "Mapped to the range [0,1]" means that 
a pixel value x which is BOTH > (maximum - Q)/255 AND <: 1 .0 becomes: 
(x- (maximum - Q)/255y(1 - (maximum - Q)fc55)). 

The resulting output image values are thresholded at g) to produce a binary image: all 
output image pixel values less than a threshold value of 0.35 are set to zero, and all other 
pixel values are set to 1 . The threshold of 0.35 was arrived at experimentally using trial 
Images. An image 70 resulting from g) is shown in Figure 5: it is equivalent to the image 
50 with -white island 0 areas 72 shown cross-hatohed. 

Referring to Figure 4 once more, the image 60 resulting from step e) contains black 
objects 61 among which there are dark tubule boundaries with respective fined (darkened) 
inner regions (originally white), all mariced as a single object of pixel 1 values and in a 
background of 0s. The Image 70 resulting from step g) contains regions 72 of pixel value 1 
which corresponding to lighter regions 52/55 of the grayscales image 50. The image 70 
also contains 0s in positions corresponding to darker regions 51/53 of the image 50. At h), 
a logical AND operation Is now carried out between each pixel In the Image 60 with the 
respective corresponding pixel in the same position In the other image 70. The AND 
operation is equivalent in this instance to multiplication because a one-bit AND operation 
gives the product of two input bits. The output of the AND operation is 1 in each case 
where both ANDed pixels are 1, and is 0 otherwise. Also at h), the result of the AND 
operation is emoothed. 
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Figure 6 shows a binary image 80 resulting from the AND operation between the images 
60 and 70 and subsequent smoothing at h). The image 80 has 1s only at locations 81 
(shown cross-hatohed) corresponding (see image 50) to central holes such as 52 in 
potential tubules 51. This has the effect of removing objects which correspond in the 
5 original greyscales image SO to isolated whits islands (usually fat cells) which are not 
surrounded by a dark boundary. 

At 36 a respective connected component labelling (CCL) operation is applied to each of 
the images 60 and 80 resulting from e) and h): CCL is a known image processing 
technique (sometimes referred to as *blob colouring 1 ) published by Wetie R„ Zamperoniu 

10 P., 'Handbook of Image Processing Operators', John Wiley & Sons. 1 998. and Rosenfeld 
A., Kak AC, 'Digital Picture Processing', vols, 1 & 2, Academic Press, New York, 1982. It 
gives different numerical labels to objects (blobs) in a binary image containing 0s and 1s 
only, objects being groups of contiguous or connected pixels of 1 : each object is assigned 
a number (label) different to others to enable it to be distinguished. CCL labels objects 

15 with numbers beginning with 1, so the numbers of the highest numbered objects in the 
images 60 and 80 are respectively the number of objects which might potentially be 
tubules and the number of holes previously within dark areas. 



A tubule may contain one or more holes, and this is required to be detected to avoid an 
incorrect tubule count. At 38, each pixel in the h) Image 80 is multiplied by the 

20 corresponding pixel in the same location in the CCL of image 60, which is a colour image 
when displayed on a colour monitor because CCL gives different colours to different 
objects. Figure 7 shows an image 90 resulting from multiplication at 38: the image 90 
does not have an image feature corresponding to the lower left hand blob in Figure 4. 
because it has been eliminated by multiplication by 0s in corresponding locations in the 

25 image 80. The image 90 retains features 91 , 92 and 93a to 93c corresponding to tubule 
holes such as 62 in the initial image 50. In this example images were processed using 
computer software referred to as "Mattab®" produced by Mathworks Inc., an American 
corporation. A Matlab function "ismember" Is used to identify holes 91 and 92 associated 
with respective single tubules that have different labels, and holes 93a to 93c all 

30 associated with the same tubule that have the same label albeit different to those of holes 
91 and 92. Differences In labelling and colouring is indicated in Figure 7 by differing 
shading, i.e. hole 91 Is unshaded, hole 92 is dotted and like-labelled holes 93a to 93c are 
all cross-hatched. The number of different shadings in Figure 9 reflects the number of 
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tubules corrected for multiple holes in any tubule* 

The areas of the objects and holes labelled are also obtained using a Matlab function 
"And" In order to compute the number of pixels per coloured object or hole. This gives the 
total object area in each case for the four objects 61 In Figure 4 (of which the lower left 
5 object 61 Is later discarded), and the hole area for each of the holes 91 to 93 in Hgure 7. 

The following parameters are now derived: 

Af a Number of objects containing one or more holes (fn image 90 f N » 3) 
NOB = Number of objects with and without holes (NOB = 4 in Image 60) 

total—holes cwea , _ 

SUHF= Toml _ ob j ecrs total area = mt[Q ^ total surface area of holes to total surface 

1 0 area of objects Including their holes. 

PERCENT o N/NOB = ratio of number of objects with holes to total number of 

objects with and without holes 

RAT/ ° = ab^wTarea ' 1or eaoh ^ject r the ratio of the area of its hole(s) to its area 

including its hole(s): RATIO is large for objects (tubules) with relatively 
15 large holes and small for objects with relatively srnan holes compared to 

object size. 

T = Number of objects containing significantly large holes, i.e. objects having 
R477O>0.09 

The threshold used to determine T discriminates against small holes unlikely to 
20 correspond to tubules. 

At 42 the parameters obtained above are used to grade tubules detected in an image, and 
to derive a score with a value of 1, 2 or 3: the score is derived with reference to 
prearranged thresholds obtained by analysing a set of images which a pathologist has 
graded. The thresholding operation is performed using independent tests: the results of 
25 these tests are subsequently combined to yield a composite mitosis score. Since the teste 
are all independent, any one on its own or combination of two or more can be used to 
provide a mitosis score, but use of all five tests to provide a composite score produces 
better results. 
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Testl : PERCENT Is compared to 1 2 and 20. A high PERCENT >20 has been observed to 
characterise score 1 images, while a low PERCENT <12 characterises score 3 images; 
otherwise, i.e. if 12 £ PERCENT S 20, an Image score of 2 is indicated. 

Test2: T = Number of objects containing medium to large holes, I.e. for which 
5 RATIO > 0.09, is compared to threshold values of 2 and S: T< 2, i.e. 7= 1 or 0, indicates 
an Image score of 3, whereas 7= 5 or greater indicates score 1; otherwise, i.e. if 7= 2, 3 
or 4 y the score is more likely to be 2. 

Tesl3: RATIO is large for objects {tubules) with relatively Jarge holes and small for objects 
with relatively small holes compared to object size. RATIO is compared to threshold 
10 values of 0.07 and 0.03. When holes are large corresponding to image soore 1 , RATIO is 
likely to be higher than 0.07; when holes are small corresponding to image score 3, 
RATIO is likely to be below 0.03. Otherwise, i.e. if 0.03 <: RATIO <> 0.07, the indicated 
image score is 2. 

Test* N = Number of objects containing one or more holes (in image 90, N = 3) 
1 5 corresponding to tubules. N is compared to 20 and 1 1 r N > 20 indicates a score 1 image 
and N < 1 1 a score 3 image; otheiwise i.e. if 1 1 £ N £ 20, the indicated image soore is 2. 

Tests: SURF: If SURF Is greater than 0.001, the total area of holes is large indicating an 
image score of 1; if SURF is less than or equal to 0.0002, the total area of holes is small 
indicating an image score of 3; otherwise, i.e. if SURF is greater than 0.0002 but not 
20 greater than 0.001 , an image score of 2 is indicated. 

If PERCENT is greater than or equal to 600 the other teste are ignored and the image 
score i& graded as 3. If PERCENT Is less than 600, the final mitosis score for an image is 
taken as the median value of five scores obtained respectively from the above five tests. 
When two Images are used, two results for each test are obtained: the mean of these two 
25 results fs computed and used with corresponding mean values from the other tests to 
derive a median value over the test results. 

The invention was tested on a set of 206 images acquired at low magnification XtO: the 
results were 44 images at score 1 , 24 Images at score 2 T 1 38 Images at score 3 were 
gathered The thresholds thereafter were set after a thorough observation of the image 
30 data obtained (grouping, clustering...) and reaching a compromise. It is important to note 
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that those thresholds may not be appropriate to another set of data because of staining 
differences. Furthermore thoss thresholds might be biased due to the fact that the image 
dataset contained a disproportionately large number of score 3 images. 

The percentage and numbers of images graded by tha classifier process of the invention 
in agreement with a pathologist's score was computed as well as the percentage and 
numbers of images where agreement was lacking. The resufts are tabulated below. 



Pathologist -» 


Score f 


Score 2 


Score 3 


Classifier i 








Score 1 


70.4% 


4.2 % 


5.1 % 


Score 2 


18.2% 


70.8 % 


21.7% 


Score 3 


11 .4 % 


25% 


73.2 % 



Pathologist 


Score I 


Score 2 


Score 3 


Classifier J, 








Score I 


31 


1 


7 


Score 2 


8 


17 


30 


Score 3 


5 


6 


101 



Mitosfs gradlngs were calculated ov&r 206 available images and compared with a 
pathologist's scores with the following resufts; 

- 72.3 % (1 49) images were graded by the classifier process of the invention 
with a ©core In agreement wfth a pathologist's score; 

- 21 .8 % (45) images were graded by the classifier process of the invention with 
a score differing by 1 from a pathologist's score; and 

• 5.8 % (12) (mages were graded by the classifier process of the invention with a 
score differing by 2 from a pathologist's score. 
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•me image dataset contained more than three times as many score 3 images [138) 
compared to score 1 images (44) and more than five times as many compared to score 2 
images (24). Best results were obtained for images scored 3 by pathologists with an 
73.2% oofrect classification; this might be attributable to the high number of images 
available enabling thresholds to be more optimised for score 3 images. A dassrfication 
which Is at least 70% correct was abhleved for images of all three scores 1 , 2 and 3. Since 
pathologists themselves are not necessarily more than 70% correct, this means that so ter 
as it is possible to do so the Invention Is verified as regards all three scores. The average 
time to compute results for an image was estimated to be in the range 20 to 40 seconds. 

The processing of digital image data in the tubules process of the Invention can clearly be 
implemented by an appropriate computer program recorded on a carrier medium and 
running on a conventional computer system. Such a program is straightforward for a 
skilled programmer to implement without requiring invention, because a number of the 
processing functions are commercially available as indicated, and others are well known 
computational procedures. Such a program and computer system will therefore not be 
described further. 
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CLAIMS 

1. A method of grading tubules In a first image (50) of a histological slide 
characterised in that it has the steps of: 

a) providing a second image (50) of first objects (61) in the first image (50) 
which are sufficiently large and of appropriate pixel value characteristics at 
boundaries to correspond to tubules, 

b) providing a third image (70) of second objects (72) in the first Image (50) 
having pixel value characteristics of fat and holes within tubules, 

c) combining data from the second and third images (60 P 70) to identify 
selected second objects (81) which are within first objects (61), 

d) performing one or more of the following: 

I) counting first objects (61 ) In the first image (50) which correspond to 

tubules to provide a parameter NOB, . 
li) counting the first objects (51) having selected second objects (81) 

within them to provide a parameter N, 
sii) determining the relative area of selected second objects (81) as 

proportions of respective first objects (51) within which they are 

located to provide parameters RATIO, 

iv) determining the total area of selected second objects (81) as a 
proportion of total area of first objects (51) within which they are 
located to provide a parameter SURF, 

v) determining a parameter PERCENT = N/NOB, and 

vi) oounting the number of first objects (51) containing at least medium 
sized holes (52) to provide a parameter 7", and 

e) grading the first image's tubules (51) on the basis of the one or more 
parameters as aforesaid with reference to parameter threshold values* 

2. A method according to Claim 1 characterised in that the step of providing a second 
image (60) incorporates; 

a) theshoiding the first image (50) to provide a fourth Image retaining relatively 
darker image pixels and rejecting relatively lighter Image pixels, 

b) processing the fourth Image by applying the following steps in succession, 
the first being applied to the fourth image and each after the first being 
applied to the outcome of the respective immediately preceding step: 
inversion, morphological dilation, median filtering, hole filling and 
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morphological opening. 



3. A method according to Claim 1 characterised in thai the step of providing a third 
image (70) comprises thresholding the first image (50) to provide a binary fourth 
image In which relatively lighter image pfxeis have a different binary value to that of 
relatively darker image pixels. 



A method according to Claim 1 characterised in that the step of combining data 
from the second and third images (60, 70) comprises multiplying pixels in the 
seoond image (60) by or ANDing them with respective corresponding pixBla 
located in like positions the third image (70). 

A method according to Claim 1 characterised in that the step of grading the first 
image's tubules (51) employs parameter threshold vsIubs set to obtain a grading 
comparable with that obtainable by an appropriate medical expert. 

A method of grading tubules in a first Image (50) of a histological slide 
characterised in that H has the steps of: 

a) providing a second image (60) of first objects (61) In the first image (50) 
which are sufficiently large and of appropriate pixel value characteristics at 
boundaries to correspond to tubules and counting them to provids a 
parameter NOB, 

b) providing a third image (70) of second objects (72) in the first image (50) 
having pixel value characteristics of fat and holes within tubules, 

c) combining data from the second and third images (60, 70) to identify 
selected seoond objects (81) which are within first objects (61), 

d) counting the first objects (51) having selected second objects (81) within 
them to provide a parameter N, 

e) determining the relative area of selected second objects (81) as proportions 
of respective first objects (51) within which they are located to provide 
parameters RATIO, 

f) determining the total area of selected second objects (81) as a proportion 
of total area of first objects (51) within which they are located to provide a 
parameter SURF, 

g) determining a parameter PERCENT = N/NOB, 

h) counting the number of first objects (51) containing at least medium sized 
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holes (52) to provide a parameter T 9 and 
i) grading the first Image's tubules (51) on the basis of the first image T s 
parameters as aforesaid with reference to parameter threshold values. 

7. Computer apparatus for grading tubules in a first image (50) of a histological slide 
characterised in that it Is programmed to: 

a) compute a second image (SO) of first objects (61) fn the first image (50) 
which are sufficient!/ large and of appropriate pixel value characteristics at 
boundaries to correspond to tubutes, 

b) compute a third image (70) of second objects (72) in the first image (50) 
having pixel value characteristics of fat and holes within tubufes, 

c) combine data from the second and third images (60, 70) to identify selected 
second objects (81 ) which are within first objects (61), 

d) Implement one or more of the following: 

i) counting first objects (61 ) in the first image (50) which correspond to 
tubules to provide a parameter NOB, 

ii) counting the first objects (51) having selected second objects (81) 
within them to provide a parameter N t 

Hi) determining the relative area of selected second objects (81) as 

proportions of respective first objects (51) within which they are 

located to provide parameters RATIO, 
fv) determining the total area of selected second objects (81) as a 

proportion of total area of first objects (51) within which they are 

located to provide a parameter SURF, 
v) determining a parameter PERCENT '= N/NQB, and 
vl) counting the number of first objects (51) containing at least medium 

sized holes (52) to provide a parameter T f and 

e) grade the first image's tubuies (51) on the basis of the one or more 
parameters as aforesaid with reference to parameter threshold values, 

8. Computer apparatus according to Claim 7 characterised in that it is programmed 
to provide a second image (60) by: 

a) thesholding the first Image (50) to provide a fourth image retaining relatively 
darker Image pixels and rejecting relatively lighter image pixels, 

b) processing the fourth image by applying the following steps in succession, 
the first being applied to the fourth image and each after the first being. 
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applied , to the outcome of the respective immediately preceding step: 
inversion, morphological dilation, median filtering, hole filling and 
moiphologioal opening. 

9. Computer apparatus according to Claim 7 characterised in that it is programmed 
to provide a third image (70) by thresholding the first image ($0) to provide a binary 
fourth image in which relatively lighter image pbcelB have a different binary value to 
that of relatively darker image pixels. 

10. Computer apparatus according to Claim 7 characterised in that it is programmed 
to combine data from the second and third images (60, 70) by multiplying pixels in 
the second Image (60) by or AN Ding them with respective corresponding pixels 
located in like positions the third imagB (70). 

11. Computer apparatus aocording to Claim 7 characterised in that it is programmed 
to grade the first image's tubules (61) with parameter threshold values set to obtain 
a grading comparable with that obtainable by an appropriate medical expert 

1 2. Computer apparatus for grading tubules in a first image (50) of a histological sl\d& 
characterised in that it is programmed to: 

a) provide a second image (60) of first objects (61) in the first image (50) 
which are sufficiently large and of appropriate pixel value characteristics at 
boundaries to correspond to tubules and counting them to provide a 
parameter NOB, 

b) provide a third image (70) of second objects (72) in the first image (50) 
having pixel value characteristics of fat and holes within tubules, 

o) combine data from the second and third images (60, 70) to identify selected 
second objects (81) which are within first objects (61), 

d) count the first objects (51) having selected second objects (81 ) within thsm 
to provide a parameter N, 

e) determine the relative area of selected seoond objects (81 ) as proportions 
of respective first objects (51) within which they are located to provide 
parameters RATIO, 

f) determine the total area of selected second objects (81) as a proportion of 
total area of first objeots (51) Within which they are located to provide a 
parameter SURF, 
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g) determine a parameter PERCENT = N/NOB, 

h) count the number of first objects (51) containing at least medium sized 
holes (52) to provide a parameter T, and 

I) grade the first Image's tubules (51) on the basis of the first image's 
parameters as aforesaid with reference to parameter threshold values. 

13. A computer program for use in grading tubules in a first image (50) of a histological 
slide characterised In that it contains instructions for controlling computer 
apparatus to: 

a) compute a second Image (60) of first objects (61) in the first image (50) 
which are sufficiently iaige and of appropriate pixel value characteristics at 
boundaries to correspond to tubules, 

b) compute a third image (70) of second objects (72) in the first image (SO) 
having pixel value characteristics of fat and holes within tubules, 

o) combine data from the second and third images (60, 70) to identify selected 
second objects (81 ) which are within first objects (61 ), 

d) Implement one or more of the following: 

r) counting first objects (61) in the first Image (50) which correspond to 
tubules to provide a parameter NOB t 

ii) counting the first objects (51) having selected second objects (8 1 > 
within them to provide a parameter N, 

iii) determining the relative area of selected second objects (81) aS 
proportions of respective first objects (51) within whioh they e* re 
located to provide parameters RA TIO, * 

iv) determining the total area of selected second objects (81) a s a 
proportion of total area of first objects (51) within which thay are 
located to provide a parameter SURF, 

V) determining a parameter PERCENT = N/NOB, and 
vi) counting the number of first objects (51 ) containing at least rne<i* um 
sized holes (52) to provide a parameter T t and 

e) grade the first image's tubules (51) on the basis of the one or more 
parameters as aforesaid with reference to parameter threshold values- 

14. A computer program according to Claim 13 characterised in tnat rt cont^» ns 
Instructions for provision of a second image (60) by: 

a) thesholding the first image (50) to provide a fourth image retaining rela*» ve, y 
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darker Image pixels and rejecting relatively lighter image pixels, 
b) processing the fourth image by applying the following steps In sucoession, 
the first being applied to the fourth image and each after the fir st e 
applied to the outcome of th9 respective Immediately preceding 
inversion, morphological dilation, median filtering, hole 
morphological opening. 



filling a nd 



15, A computer program according to Claim 13 characterised in that it °° nt ^! 0 j 
instructions for provision of a third Image (70) by thresholding the *' ret lrnase 

to provide a binary fourth image In which relatively lighter image p!* e!s haV0 
different binary value to that of relatively darker image pixels. 

16. A computer program according to Claim 13 characterised in that it oort ® lr * 

<= <GO 70) °y 

instructions for combining data from the second and third images ^ ^ 
multiplying pixels in the second image (60) by or ANDing them with resP e 
corresponding pixels located in like positions the third image (70). 

it contain® 

17. A computer program according to Claim 13 characterised in that 
Instructions for grading the first image's tubules (51) with parameter r 
values set to obtain a grading comparable with that obtainable by ^ n appr0 
medical expert. 

18. A computer program for grading tubules In a first image (50) of a his* olo9icgtl 
characterised in that it contains instructions for controlling computer apP aratuS ^ 

a) provide a second image (60) of first objects (61) in the lm8 ®^ 
which are sufficiently large and of appropriate pixel value cft& ra e a 
boundaries to correspond to tubules and counting then* *° 
parameter NOB, (SO ) 

b) provide a third image (70) of second objBcts (72) in the "h*** 

having pixel value characteristics of fat and holes within tufc>u ,0S * le ot« d 
o) combine data from the second and third images (60, 70) to ide" 1 ^ s 



second objects (81 ) which are within first objects (61 ). 

d) count the first objects (51 ) having selected second objects 
to provide a parameter N, ar& <yns 

e) determine the relative area of selected second objeots (Sri > 08 p rtrC >vide 



of respective first objects (51) within whioh they are 



lp o^ted to Pf° 
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parameters RATIO, 

f) determine the total area of selected second objects (81) as a proportion of 
total area of first objects (51) wifliln which they are located to provide a 
parameter SURF, 

g) determine a parameter PERCENT =N/NOB, 

h) count the number of first objects (51) containing at least medium sized 
holes (52) to provide a parameter T, and 

I) grade the first image's tubutes (51) on the basfs of the first image's 
parameters as aforesaid wrth reference to parameter threshold values. 
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ABSTRACT 

A method of grading tubules in a first histological slide image (50) derives a second image 
(60) of objects (61) in the first image (50) with boundary characteristics corresponding to 
tubules. It also derives a third image (70) of second objects (72) in the first image (60) 
having pixel value characteristic of fat and holes within tubules. It combines data from the 
second and third images (60. 70) to identify holes (81) within tubules (61) and determines 
the relative areas of holes (81) as proportions of their tubules (51) to provide ratios, 
Individual tubule ratios and an overall ratio for all holes and tubules collectively. The 
number of tubules (51) containing appreciably sized holes (52) is counted. Tubules (51) 
are graded by thresholding based on individual and overall tubule/hole area ratios, 
tubule/object proportion, tubule number and number of tubules with appreciably sized 
holes. Thresholds are derived from image gradation by an appropriate medical expert. 
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